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Abstract 

Designers of out- the- window visual systems face 
a challenge when attempting to simulate the outside 
world as viewed from a cockpit. Many 
methodologies have been developed and adopted to 
aid in the depiction of particular scene features, or 
levels of static image detail. However, because 
aircraft move, it is necessary to also consider the 
quality of the motion in the simulated visual scene. 
When motion is introduced in the simulated visual 
scene, perceptual artifacts can become apparent. A 
particular artifact related to image motion, spatio- 
temporal aliasing, will be addressed. The causes of 
spatio-temporal aliasing will be discussed, and 
current knowledge regarding the impact of these 
artifacts on both motion perception and simulator 
task performance will be reviewed. Methods of 
reducing the impact of this artifact are also addressed. 

Background 

Aliasing occurs when a continuous signal is 
digitized using a sampling scheme that is too sparse 
(i.e., at an inadequately low frequency). This can be 
a spatially-varying signal, where sampling frequency 
refers to the spacing of sampling locations (as in the 
2-D display), or a temp orally- varying signal where 
sampling frequency refers to the sampling period. 
When the signal is periodic (repeating, either in time 
or spatial direction as in a pattern), and the sampling 
interval drops below the Nyquist frequency 1 , the 
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sampled signal will generally not faithfully reproduce 
the original signal, and additionally will typically 
exhibit frequency content that is dramatically 
different than the original signal. Figs. 1-3 depict the 
effect of sampling at rates above, at, and below the 
Nyquist frequency for time-varying signals. 
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Fig. 1 Time varying continuous sinusoidal signal 
sampled at a rate above the Nyquist frequency (top), 
and sampled reconstruction (bottom). The 
reconstructed signal is a good representation of the 
original signal. 


1 The Nyquist frequency is two-times the bandwidth of the 
signal being sampled. For example, the Nyquist frequency 
for a signal consisting of a sine wave with a 30-Hz 
frequency, the Nyquist frequency is 60 Hz. 
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Fig. 2 Time varying continuous sinusoidal signal 
sampled at the Nyquist frequency (twice the signal 
frequency) (top), and sampled reconstruction 
(bottom). While the reconstructed signal is not a 
perfect reproduction of the original signal, it retains 
the essential frequency characteristic of the original 
signal. 
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Fig. 3 Time varying continuous sinusoidal signal 
sampled at a rate below the Nyquist frequency (top), 
and sampled reconstruction (bottom). The 
reconstructed signal is not an accurate representation 
of the original signal, and in fact appears to have a 
different, lower fundamental frequency than the 
original signal. 


Aliasing can also occur in a sampled two- 
dimensional signal, such as an image. This effect can 
be easily observed when viewing a texture in a 3-D 
scene without the appropriate level of filtering or 
mip-mapping; the resulting perceived texture can 
appear to be have a much lower spatial frequency, or 
the periodicity that might originally be present in a 
texture could appear to be highly irregular or 
incoherent (Fig. 4). 



Fig. 4 Runway image applied as a texture with 
antialiasing off (left) and on (right). Significant 
spatial aliasing is evident on the left, particularly in the 
foreground. 

Spatio-temporal aliasing is associated with the 
temporal sampling of a moving image. Perhaps the 
best known example of this is the wagon- wheel 
effect, observed in camera images of a spoked wheel 
(Fig. 5). 



Fig. 5 Example of spatio-temporal aliasing. In (a), if 
the spoked wheel rotates a small amount, the 
perceived direction of motion will be in the true 
direction of motion; the brain connects the motion of 
the spokes in the proper sequence. In (b), the wheel is 
rotating more than half of the angle between the 
spokes per frame; instead of perceiving motion in the 
correct direction (black arrow) motion appears to be 
occurring in the opposite direction (red arrow) because 
of the smaller angular offset to the next spoke. 

Although the wagon- wheel example is a special 
case of a periodic or repeating visual image (from the 
symmetry of the wheel), spatio-temporal aliasing can 
occur with any image sequence depending on the 
sampling rate and image resolution. 

Analysis of spatio-temporal aliasing, and often 
explanation of the phenomenon and the resulting 
effects, are frequently done in the frequency domain, 
based on periodic or repeating features in the image 
(as shown in the previous examples). Although these 



techniques can very accurately represent the effects 
of temporal sampling of a time-varying, two- 
dimensional spatial image, our perceptions occur in 
the time-domain. Specific cases in which we can 
easily observe spatio-temporal aliasing in visual 
simulation include sampling moderate image motions 
at too low a rate 2 , or sampling very fast motion at a 
moderate rate 3 . 

Perceptual Consequences 

Watson, Ahumada, and Farrell (1986) proposed 
an analytical method to predict when a human 
observer would detect spatio-temporal aliasing, 
considering both CRT-like displays (very short 
duration) and LCD-like displays (sample and hold). 
Given the limits of human temporal sensitivity 
(typically 60 Hz), and limits of spatial acuity 
(typically 30 to 60 cycles/deg), the method can be 
used to predict the “critical” sampling frequency, the 
frequency at which sampled motion could be 
distinguished from continuous motion. The critical 
frequency can be determined with the following 
formula: 

oo c = oi>i + rui (Eq. 1) 

where oo c is the critical sampling frequency, oe>i is the 
temporal sensitivity (in cycles/sec or Hz), r is the 
speed of the image motion (in degrees/sec), ui is the 
limit of spatial acuity (in cycles/deg). When the 
image itself is limited in spatial frequency (i.e. less 
than eye-limiting resolution), the critical frequency 
becomes 

oa c = ol>i + r Ub (Eq. 2) 

where Ub represents highest spatial frequency content 
of the image. 

In the equations above, critical sampling 
frequency (refresh rate at which spatio-temporal 
aliasing becomes noticeable) is determined as a 
function of human temporal sensitivity, image 
motion, and image spatial resolution. In practice, 
refresh rate is not a variable; it is a fixed feature of a 
display. If the sampling frequency is known, Eq. (3) 
predicts the level of image motion at which spatio- 
temporal aliasing becomes noticeable: 

r = (co c - coi)/ u b (Eq. 3) 


2 Such as is experienced when an image generator regularly 
runs over frames and updates every two refresh cycles. 

3 Such as is experienced in depicting high rotational rates, 
as in fighter aircraft, or when moving rapidly in close 
proximity to a planar surface such as a runway. 


Solving for image motion (r) for a display 
refresh rate (oo c ) of 60 Hz, and a typical human 
temporal frequency limit (ooi) of 60 Hz, we find that 
any image motion is expected to produce spatio- 
temporal aliasing (r = 0). This finding is not 
completely consistent with flight simulation 
experience, where refresh rates of 60 Hz are routinely 
used with acceptable image performance. This 
discrepancy can likely be explained by considering 
that the Watson et al. (1986) analysis is based on the 
discrimination between sampled and continuous 
motion, not on assessing when spatio-temporal 
artifacts become noticeable or distracting, or 
adversely impact functionality. Additionally, the 
model assumes space-time separability (i.e., coi is 
independent of Ub) and linearity (i.e., the responses to 
each image frequency are independent from each 
other). The empirical validation of the model was 
also accomplished using essential ‘point’ features 
(i.e., a single frequency components) rather than a 
full image. Thus, known non-linear interactions 
across frequency components, such as masking, were 
not taken into consideration. 

This discrepancy stills allows for the 
examination of qualitative rather than quantitative 
effects of refresh rate and image resolution. Fig. 6, 
derived from Eq. (3), shows the relationship between 
this theoretical image motion threshold and spatial 
resolution for varying refresh rates. It is expected that 
at higher resolutions, the visibility of spatio-temporal 
artifacts will occur at lower image motions. 
Additionally, increasing refresh rate would likely 
reduce the visibility of these artifacts. 



Fig. 6 Image motion at which spatio-temporal aliasing 
is predicted to become detectable as a function of 
image spatial resolution and refresh rate. 

There are additional empirical results studying 
the effects of refresh rate on spatio-temporal artifacts 
using imagery rather than a ‘point’ feature. Kuroki, 
Nishi, Kobayashi, Oyaizu, and Yoshimura (2006) 
conducted a series of experiments examining the 
effect of refresh rate on moving image quality. Using 
a CRT display capable of achieving refresh rates of 
up to 450 Hz, they obtained subjective measures of 


motion quality as a function of refresh rate, for rates 
of 62, 125, 250, 333, and 450 frames/sec. The image 
sequences studied contained motions in the range of 
7 to 78 deg/sec. 

Subjective motion quality measures, related to 
the absence of blur and ‘jerkiness’, improved with 
increasing frame rates up to 250 fps; rates above this 
did not produce significant improvements in the 
subjective measures. Although 125 fps refresh rate 
was associated with some degradation relative to 
higher refresh rates, it was significantly better than 
the lowest refresh rate of 62 fps. 

Although this study provides some insight 
regarding the point of diminishing returns regarding 
refresh rate, the results are not directly applicable to 
real-time visual simulation image generation. The 
image sequences used in this study were captured 
using a high-speed camera (at 1000 Hz), then for 
each refresh rate, frames were generated through a 
frame- averaging process. Thus a 500 fps image 
sequence was generated by averaging two frames, 
333 fps by averaging 3 frames, etc. This effectively 
reduced the resolution or spatial frequency of the 
image sequences as a function of displayed refresh 
rate, rather than examining refresh or frame rate 
independent of the spatial resolution of the image. 
This technique introduced image blur typical of fixed 
frame rate camera images. 

Winterbottom. Gaska, Geri, and Sweet (2008) 
conducted an assessment of a laser projector capable 
of update rates up to 240 Hz (Kikuchi et al., 2008). 
Interestingly, this assessment yielded similar results 
to those reported by Kuroki et al. (2006). Image 
sequences of an F-16 aircraft moving perpendicular 
to the line-of-sight of the observers were generated 
(without simulated motion blur or frame averaging) 
to correspond to 10, 20 and 40 deg/sec aircraft 
motion (see Fig. 7). The projector was configured to 
be at a spatial resolution of 11.5 cycles/deg. 
Observers were asked to report the presence of 
motion artifacts at update rates of 60, 120, and 240 
Hz. At 60 Hz, subjects reported motion artifacts at 
all conditions. For the 120 Hz condition, motion 
artifacts were apparent only at the highest motion (40 
deg/sec). No motion artifacts were reported at the 
240 Hz update rate. 



Fig. 7 Single frame of image sequence used to assess 
spatio-temporal aliasing with high frame rate laser 
projector (top), and cropped image showing level of 
detail of F16 (uncompressed) shown below. 

These empirical results are somewhat more 
promising than the theoretical analysis presented in 
Fig. 6. It should be noted that the Watson method is 
related directly to differentiating smooth from 
temporally- samp led motion, whereas the Kuroki and 
Winterbottom empirical studies were related to the 
quality of the motion depiction. Additionally, the 
Watson analysis did not consider the eye movements 
of the observer. As will be shown, some perceptual 
artifacts of spatio-temporal aliasing are likely related 
to the eye movements of the observer. 

Eye Movements 

Humans have two distinct types of voluntary eye 
movements that they use to follow moving objects of 
interest: pursuit which smoothly moves the eyes to 
match the eye velocity to target velocity and saccades 
which ballistically correct for position offsets in order 
to keep the image of the target close to the fovea (the 
retinal locus of highest acuity). 

Pursuit is a highly evolved motor capability that 
taps into an elaborate primate brain pathway 
dedicated to motion processing (Maunsell & 
Newsome, 1987). Under a broad range of conditions, 
it is capable of generating very precise ocular control 
limited by visual speed and direction signals shared 
with perception (Kowler & McKee, 1987; Stone, 
Beutter, Eckstein, & Liston, 2008; Stone & Krauzlis, 
2003). Furthermore, the accuracy of pursuit is 
directly linked to perceptual accuracy (Beutter & 
Stone, 1998, 2000; Krukowski & Stone, 2005; Stone, 
Beutter, & Lorenceau, 2000). Typically, humans can 
generate eye speeds that nearly match target speeds 
(i.e., gains in excess of 90%) with few catch-up 



saccades necessary for target speeds up to about 30 or 
40 deg/sec and can perform effectively up to about 60 
deg/sec where saccadic tracking then begins to 
dominate (e.g., Lisberger & Westbrook, 1985; see 
also Fig. 8). 

As stated above, human motion perception has a 
spatio-temporal sensitivity that makes it vulnerable to 
perceiving spatio-temporal aliasing when the 
displayed moving image is under-sampled either 
spatially or temporally (or both). When target motion 
is under-sampled, humans perceive artifacts that 
disrupt acuity and distort spatial features of the target 
(e.g. doubling, blurring), and ability of smooth 
pursuit to continuously track moving objects is 
degraded. 

Fig. 8 demonstrates the effect of refresh rate on 
eye movements. Eye movements are shown for 
target velocities of 10 to 60 deg/sec at two refresh 
rates, 60 Hz and 120 Hz. The eye movements are 
color coded to differentiate smooth pursuit eye 
movements (blue) from saccadic eye movements 
(red). The dashed line indicates what ‘perfect’ eye 
movements would be, i.e. the fovea is maintained on 
target. As can be seen, at the 60 Hz refresh rate 
catch-up saccades typically begin to occur frequently 
at 30 deg/sec target motion. For 120 Hz refresh rate, 
catch-up saccades become more prevalent at 40 or 50 
deg/sec target motion, close to the limit of effective 
pursuit tracking. 

This increase in catch-up saccades is not only 
caused by the impoverished visual signal but also 
may contribute to perceptual artifacts commonly seen 
with spatio-temporal aliasing. At some conditions, a 
smoothly-moving image will suddenly seem to 
double or jump. Catch-up saccades produce sudden 
displacements of image features to different locations 
on the retina, which could lead to perceptual 
doubling. This effect is similar to a ‘double 
exposure’ caused by a jarred camera as, for an instant 
in time, a feature is perceived to be in two places at 
once. 

Lastly it should be noted that the sensitivity to 
spatio-temporal aliasing it dependent on whether the 
image motion is actually tracked with pursuit eye 
movements, or whether the eyes are fixated on a 
fixed point within or over the moving image. More 
specifically, although spatio-temporal aliasing may 
compromise pursuit, the pursuit response will tend to 
decrease the motion of the attended feature on the 
retina and thus act to reduce aliasing of the tracked 
feature or at least to change its qualitative properties. 
(Conversely, pursuit may actually increase the 
aliasing of peripherally viewed features which could 
adversely impact or delay responses to secondary 
tasks). Thus, the overall perceptual impact of display 
sampling will depend on whether a moving display 
feature is tracked or if eyes remain fixated on a 
stationary feature. Spatio-temporal aliasing artifacts 
were examined by Kuroki et al. (2006) in both 


fixation and pursuit. They found that spatio-temporal 
aliasing was much more evident in fixation than 
pursuit. In visual simulation, fixation is certainly the 
exception to the rule; operators are most likely 
engaged in pursuit eye movements when viewing a 
simulated out-the-window display. This provides an 
additional explanation for the apparent disconnect 
between the predictions of Watson et al. (1986) and 
common simulator applications for 60 Hz update 
rates. 



Fig. 8 Eye movement velocity for 60 Hz (top) and 120 
Hz (bottom) refresh rates, at varying target speeds. 
When the blue line is directly over the dashed line, 
pursuit eye movements are tracking the target 
accurately. When pursuit starts to fall behind (blue 
line dips below dashed line), the eye does a “catchup 
saccade” (red) to realign the eye with the target. 


The theoretical and experimental results 
described in this section all indicate frame rates on 
the order of 240 Hz are necessary to minimize 
perceptual artifacts resulting from spatio-temporal 
aliasing. This represents a daunting technical 
challenge. For visual simulation, perhaps the more 
important question is when does spatio-temporal 
aliasing create an erroneous perception of motion, 
and how does that affect task performance? 
Oculometric analysis (as illustrated in Fig. 8) may 
provide a useful objective tool for establishing the 
functionally relevant trade-offs between sampling 
rate and visuomotor performance (as opposed to 
subjective judgments of perceptual esthetics) in the 
important 90 to 240 Hz range. 

Simulator Performance Implications 

As was the case regarding the effects of spatio- 
temporal aliasing on perception, relatively little 
research has been done quantifying the effects of this 
artifact on task performance in simulators. However, 
there are several research topics that are directly 
relevant to this issue. First, many factors influencing 
vehicular control tasks are well understood, including 
the impact of velocity information on task 
performance. Secondly, research has demonstrated 
the effect of impaired velocity information on active 
control tasks which require velocity information. 
Lastly, some research has addressed the issue of 
spatio-temporal aliasing on specific simulated flight 
tasks. 

Motion Perception Requirements for Vehicular 
Control 

Tasks typically performed in a simulator span a 
continuum from very tight, continuous, closed-loop 
control, to monitoring and the execution of discrete 
actions. The first end of this continuum is commonly 
called manual control. Dating from the 1940’s, 
researchers in this engineering field have sought to 
determine what type of compensation, or control 
activity, a human operator will provide to accomplish 
a vehicular control task (comprehensive overviews of 
manual control are available in Wickens, 1987, and 
Hess, 1997). 

One finding from the field of manual control is 
that the type of compensation an operator adopts, and 
the information needed to perform that compensation, 
is a function of the vehicle being controlled, and the 
task. An extensive body of work exists which 
confirms that in some manual control tasks, the 
operator needs to detect and respond to velocity 
information to effectively accomplish the task. Thus, 
in some vehicular control situations, perception of 
visual motion is potentially critical. Particular 
vehicle control tasks that fall into this category are 
helicopter position control, spacecraft attitude and 


position control, roll control in some aircraft types, 
and lateral path control in a car when forward 
visibility is limited. 

Li, Sweet, and Stone (2005, 2006) conducted 
several experiments that demonstrated the effect of 
visual motion perception on manual control tasks 
dependent on velocity perception. Performance was 
degraded when visual motion perception was 
compromised, both through contrast and color 
variations. 

Spatio-temporal Aliasing Effects on Simulator Tasks 

Dearing, Schroeder, Sweet, and Kaiser (2001) 
conducted a study examining the effects of platform 
motion and visual scene content on performance of 
autorotative descents and landings in a Blackhawk 
helicopter. One of the hypotheses prior to the study 
was that if the ground texture resolution was too 
high, that spatio-temporal aliasing would result in 
impaired motion perception and reduced 
performance. Three texture levels were chosen, the 
highest-resolution being chosen (subjectively) based 
on the level of aliasing observed during the final 
portion of the approach. The highest resolution 
texture was associated with worse performance, in 
sink rate, than the other textures which did not create 
significant spatio-temporal aliasing. At the highest 
rate, the combination of vehicle motion, proximity to 
the textured ground plane, and texture resolution 
combined to create a perception of flashing or 
incoherent motion. The two lower texture resolutions 
appeared to move relatively smoothly and 

consistently. 

Lindholm, Sharine, and Pierce (2003) examined 
issues associated spatio-temporal aliasing in flight 
simulation applications. The authors analyzed the 
degree to which spatio-temporal aliasing would occur 
in realistic flight simulation applications, with both 
high-resolution (0.15 m/pixel) and low-resolution 
(0.56 m/pixel) textures, at simulated altitudes of 500 
and 50 m, as a function of ground speed (100 to 800 
knots). They demonstrated that at a majority of these 
conditions, a large proportion of a simulated ground 
plane would exhibit spatio-temporal aliasing, on the 
order of 80 to 90% of the ground plane for the lowest 
altitude and highest resolution texture. They report 
that studies on perceived speed during low-level 
flight had produced inconsistent results. They point 
out that proposed ultra-high resolution systems 
(Kraemer, Ashby, & Chambers, 2006) will 
experience spatio-temporal aliasing at higher 
altitudes and lower airspeeds than contemporary 
systems. 

Potential Solutions 

There are two potential solutions for reducing the 
saliency of spatio-temporal aliasing; 1) increasing 


refresh rate, and/or 2) decreasing the spatial 
resolution (or low-pass filtering) of the image. 

Refresh Rate 

As determined by Watson et. al. (1986), 
perceptible spatio-temporal aliasing is likely to occur, 
with any image motion, for frame rates of 60 Hz or 
less. Increases in frame rates above 60 Hz will 
increase the image velocities that can be displayed 
without perceptible spatio-temporal aliasing. 
Previous empirical studies have shown that while 
refresh rates of 240 Hz or higher will virtually 
eliminate perceptible spatio-temporal aliasing, refresh 
rates of 90 or 120 Hz significantly improve 
performance over the current industry standard of 60 
Hz. 

Beyond the technical complexity of driving a 
display device to refresh at higher rates there is the 
challenge of supplying image content at that rate. 
One method that will enable higher refresh rates is 
reducing scene complexity to allow rendering at the 
desired rate. Another potential approach to supplying 
image content at high frame rates is to multiplex 
additional synchronized IG channels. 

Image Resolution 

High-quality rendered images, not generated in 
real-time, typically prevent spatio-temporal aliasing 
by averaging multiple image sequences rendered at a 
series of times within the frame duration; this has the 
effect of blurring the portions of the image with 
significant motion. Real-time rendering techniques 
currently lack the capability of introducing motion 
blur to reduce or remove spatio-temporal aliasing. 
One potential solution could be the reduction of 
spatial resolution in portions of the image with a high 
image velocity. Current mip-mapping techniques 
could possibly be modified to include image velocity 
in selecting the spatial resolution of textures, 
although there are no applications of this technique 
to-date. Additionally, current multi-sample 
techniques to enable full-screen antialiasing could be 
utilized to provide real-time motion blur by 
computing samples at different time points within the 
frame interval. 

Conclusions 

Although very little research has demonstrated a 
direct correlation between spatio-temporal aliasing 
and simulator performance, current knowledge 
related to visual information requirements for closed- 
loop control can be used to infer that velocity 
perception decrements resulting from spatio-temporal 
aliasing will have an impact on simulator task 
performance. The impact is likely to be dependent 
on the particular vehicle and task, as well as other 


display characteristics. The impact of aliasing is 
expected to affect higher spatial resolution display 
systems to a greater extent than lower resolution 
systems. 

Because real-time techniques to reduce or 
eliminate spatio-temporal aliasing do not currently 
exist, the method with the most potential to reduce 
this artifact is an increase in the refresh rate. 
Although this would reduce spatio-temporal aliasing, 
it would not eliminate it in all applications and tasks, 
except possibly at rates exceeding 240Hz. It is 
possible that the saliency of any remaining spatio- 
temporal aliasing artifacts could be further reduced 
by modulating the spatial frequency content of the 
image, possibly through manipulation of spatial anti- 
aliasing techniques that are applied in real time. 

Technology advances are enabling the 

development of higher resolution systems. These 
higher resolution systems will be more vulnerable to 
spatial-temporal aliasing. The perceptual and 

performance effects of spatio-temporal aliasing are 
varied, and objective measures of the impact of this 
artifact do not currently exist although oculometric 
analysis represents a promising new approach. Much 
work remains to be done in this area. 
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